Shrinkage structure in biased regression
نویسنده
چکیده
Biased regression is an alternative to ordinary least squares (OLS) regression, especially when explanatory variables are highly correlated. In this paper, we examine the geometrical structure of the shrinkage factors of biased estimators. We show that, in most cases, shrinkage factors cannot belong to [0, 1] in all directions. We also compare the shrinkage factors of ridge regression (RR), principal component regression (PCR) and partial least squares regression (PLSR) in the orthogonal directions obtained by the signal-to-noise ratio (SNR) algorithm. In these directions, we find that PLSR and RR behave well whereas shrinkage factors of PCR have an erratic behaviour.
منابع مشابه
Liu Estimates and Influence Analysis in Regression Models with Stochastic Linear Restrictions and AR (1) Errors
In the linear regression models with AR (1) error structure when collinearity exists, stochastic linear restrictions or modifications of biased estimators (including Liu estimators) can be used to reduce the estimated variance of the regression coefficients estimates. In this paper, the combination of the biased Liu estimator and stochastic linear restrictions estimator is considered to overcom...
متن کاملGeneralized Ridge Regression Estimator in Semiparametric Regression Models
In the context of ridge regression, the estimation of ridge (shrinkage) parameter plays an important role in analyzing data. Many efforts have been put to develop skills and methods of computing shrinkage estimators for different full-parametric ridge regression approaches, using eigenvalues. However, the estimation of shrinkage parameter is neglected for semiparametric regression models. The m...
متن کاملFinding Optimal Value for the Shrinkage Parameter in Ridge Regression via Particle Swarm Optimization
A multiple regression model has got the standard assumptions. If the data can not satisfy these assumptions some problems which have some serious undesired effects on the parameter estimates arise. One of the problems is called multicollinearity which means that there is a nearly perfect linear relationship between explanatory variables used in a multiple regression model. This undesirable prob...
متن کاملResponse to the comment: Variable selection should be blinded to the outcome.
When multivariate models are used for confounder adjustment, all potential confounders should be included in the model, or if not, their selection should be blinded to the outcome. Other methods, such as “pre-filtering” variables based on their univariate association with the outcome may give rise to biased regression coefficients, biased standard errors, biased confidence intervals, misspe...
متن کاملShrinkage Estimation and Prediction for Time Series
For independent samples, shrinkage estimation theory has been developed systematically. Although shrinkage estimators are biased, they improve the MSE of unbiased ones. In view of this, we will develop shrinkage estimation theory and prediction for dependent samples. First, we propose a shrinkage estimator for the coefficients of AR model, which improves the MSE of the least squares estimator. ...
متن کامل